Repository navigation
docs: docker.yml's Docker secrets must be mapped by the caller - #21
Conversation
docker.yml logs in from a release-environment job, which receives an environment secret only when the caller maps it by name, as for BLS_RELEASE_PRIVATE_KEY. The old 'Never passed by a caller' description led two callers to drop the mapping; their first release would have stopped at the Docker Hub login.
en0f
left a comment
There was a problem hiding this comment.
Reviewed at b6a5038.
🟢 One instruction for one mechanism
DOCKER_USERNAME and DOCKER_TOKEN now carry exactly the text BLS_RELEASE_PRIVATE_KEY has in release-check.yml and publish.yml, so a caller gets the same instruction whichever of the three workflows they read. That closes the gap left when 2ecc0d1 fixed the other two copies and docker.yml kept "Never passed by a caller". The premise holds both ways in this repository's own history: the first release at 5c7ee0e, which mapped nothing, failed check / credentials at create-github-app-token (run 40890779963), and the mapped v1.3.0 release passed it (run 43443400874).
🟢 The README list is now complete
Every secret read inside an environment: release job of a reusable workflow here is now named with its workflow under "Using a workflow". crates.yml's release job reads none, and docker.yml reads the Docker secrets only in its release-environment build job, so the paragraph's mechanism applies to them as written.
Reproduce this review
# save the script below as verify-21.sh, then:
bash verify-21.sh blacklanternsecurity/release-workflows b6a5038643a3b77841bc42414f047efa23559b73
# expected: SUMMARY 7 PASS, 2 FAIL, 0 SKIP, 0 BROKEN
# V11 and V12 fail by design: V11 records that 2ecc0d1 missed this copy, V12 that no test keeps the list in sync.
# control: bash verify-21.sh blacklanternsecurity/release-workflows eff10b0f2a2cc538ff21600707c6a572da210a6d -> V01=FAIL V02=FAIL V03=FAIL V04=FAIL#!/usr/bin/env bash
# verify-21.sh: reproducible checks for blacklanternsecurity/release-workflows#21
# Usage: bash verify-21.sh <owner/repo> <sha> [<control-patch>]
# Changes nothing outside a mktemp directory and nothing on GitHub (PR-VF-02).
#
# CONTROL: sha eff10b0f2a2cc538ff21600707c6a572da210a6d
# CONTROL-EXPECT: V01=FAIL V02=FAIL V03=FAIL V04=FAIL
# AUTHOR-CODE: none
set -uo pipefail
REPO="${1:?owner/repo}"; SHA="${2:?sha}"; PATCH="${3:-}"
PASS=0; FAIL=0; SKIP=0; BROKEN=0
pass() { echo "PASS $1 $2"; PASS=$((PASS+1)); }
fail() { echo "FAIL $1 $2 -- $3"; FAIL=$((FAIL+1)); }
skip() { echo "SKIP $1 $2 -- $3"; SKIP=$((SKIP+1)); }
broken() { echo "BROKEN $1 $2 -- $3"; BROKEN=$((BROKEN+1)); }
summary() { echo "SUMMARY $PASS PASS, $FAIL FAIL, $SKIP SKIP, $BROKEN BROKEN"; [ "$FAIL" -eq 0 ] && [ "$BROKEN" -eq 0 ]; }
echo "script_sha256 $(shasum -a 256 "$0" | cut -d' ' -f1)"
echo "git $(git --version | cut -d' ' -f3)"
echo "python3 $(python3 -c 'import platform;print(platform.python_version())')"
echo "gh $(gh --version | head -1 | cut -d' ' -f3)"
WORK="$(mktemp -d)"; trap 'rm -rf "$WORK"' EXIT
export GIT_CONFIG_GLOBAL=/dev/null GIT_CONFIG_NOSYSTEM=1 GIT_TERMINAL_PROMPT=0
GIT=(git -c credential.helper= -c 'credential.helper=!gh auth git-credential')
"${GIT[@]}" clone --quiet --filter=blob:none --no-checkout "https://github.com/$REPO.git" "$WORK/src" || { broken V00 "clone" "clone failed"; summary; exit 1; }
cd "$WORK/src"
"${GIT[@]}" cat-file -e "$SHA^{commit}" 2>/dev/null || "${GIT[@]}" fetch --quiet origin "+refs/pull/*/head:refs/remotes/pr/*" || true
"${GIT[@]}" checkout --quiet --detach "$SHA" || { broken V00 "checkout $SHA" "checkout failed"; summary; exit 1; }
[ "$(git rev-parse HEAD)" = "$SHA" ] || { broken V00 "HEAD is $SHA" "HEAD mismatch"; summary; exit 1; }
if [ -n "$PATCH" ]; then git apply "$PATCH" || { broken V00 "apply control patch" "git apply failed"; summary; exit 1; }; fi
# Description of a workflow_call secret, folded the way YAML '>-' or a plain scalar folds it.
# Stdlib only: reads the indented block under '<NAME>:' in 'secrets:'.
desc() {
python3 -I - "$1" "$2" <<'PY'
import re, sys
path, name = sys.argv[1], sys.argv[2]
lines = open(path).read().splitlines()
for i, l in enumerate(lines):
m = re.match(r'^(\s+)' + re.escape(name) + r':\s*$', l)
if not m:
continue
ind = len(m.group(1))
for j in range(i + 1, len(lines)):
lj = lines[j]
if lj.strip() and len(lj) - len(lj.lstrip()) <= ind:
break
d = re.match(r'^\s+description:\s*(.*)$', lj)
if d:
v = d.group(1).strip()
if v in ('>-', '>', '|', '|-'):
k = len(lj) - len(lj.lstrip())
parts = []
for t in lines[j + 1:]:
if t.strip() and len(t) - len(t.lstrip()) <= k:
break
parts.append(t.strip())
v = ' '.join(p for p in parts if p)
print(v)
sys.exit(0)
sys.exit(1)
PY
}
DW=.github/workflows/docker.yml
# V01: docker.yml no longer tells callers its Docker secrets are never passed.
if [ -f "$DW" ] && ! grep -qi 'never passed by a caller' "$DW"; then
pass V01 "docker.yml's DOCKER_USERNAME/DOCKER_TOKEN descriptions don't say 'Never passed by a caller'"
else
fail V01 "docker.yml's DOCKER_USERNAME/DOCKER_TOKEN descriptions don't say 'Never passed by a caller'" "$(grep -n -i 'never passed by a caller' "$DW" | head -2 | tr '\n' ' ')"
fi
# V02: both Docker secret descriptions equal the App key's in release-check.yml and publish.yml.
RC="$(desc .github/workflows/release-check.yml BLS_RELEASE_PRIVATE_KEY)"; PB="$(desc .github/workflows/publish.yml BLS_RELEASE_PRIVATE_KEY)"
DU="$(desc "$DW" DOCKER_USERNAME)"; DT="$(desc "$DW" DOCKER_TOKEN)"
if [ -z "$RC" ] || [ -z "$PB" ]; then
broken V02 "Docker secret descriptions match the App key's" "could not read BLS_RELEASE_PRIVATE_KEY description"
elif [ "$RC" = "$PB" ] && [ "$DU" = "$RC" ] && [ "$DT" = "$RC" ]; then
pass V02 "DOCKER_USERNAME and DOCKER_TOKEN descriptions equal BLS_RELEASE_PRIVATE_KEY's in release-check.yml and publish.yml"
else
fail V02 "DOCKER_USERNAME and DOCKER_TOKEN descriptions equal BLS_RELEASE_PRIVATE_KEY's in release-check.yml and publish.yml" "docker: '$DU'"
fi
# V03: README's environment-secret paragraph names docker.yml and both Docker secrets.
PARA="$(awk 'BEGIN{RS="";ORS="\n\n"} /environment secret only when the caller passes it/' README.md)"
if [ -z "$PARA" ]; then
broken V03 "README environment-secret paragraph names docker.yml's secrets" "paragraph not found"
elif grep -q 'docker.yml' <<<"$PARA" && grep -q 'DOCKER_USERNAME' <<<"$PARA" && grep -q 'DOCKER_TOKEN' <<<"$PARA"; then
pass V03 "README environment-secret paragraph names docker.yml, DOCKER_USERNAME and DOCKER_TOKEN"
else
fail V03 "README environment-secret paragraph names docker.yml, DOCKER_USERNAME and DOCKER_TOKEN" "not named"
fi
# V04: every secret read inside an 'environment: release' job of any reusable workflow is named,
# with its workflow, in README's "Using a workflow" section (snippet + paragraph): nothing a caller
# must map is left out.
SECTION="$(awk '/^## Using a workflow/{f=1;print;next} /^## /{f=0} f' README.md)"
MISSING="$(python3 -I - "$SECTION" <<'PY'
import glob, re, sys
para = sys.argv[1]
need = set()
for path in sorted(glob.glob('.github/workflows/*.yml')):
src = open(path).read()
if 'workflow_call:' not in src:
continue
lines = src.splitlines()
try:
start = lines.index('jobs:')
except ValueError:
continue
job, buf = None, []
def flush():
if job and any(re.match(r'^\s+environment:\s*release\s*$', b) for b in buf):
for b in buf:
for s in re.findall(r'secrets\.([A-Za-z0-9_-]+)', b):
if s != 'GITHUB_TOKEN':
need.add((path.split('/')[-1], s))
for l in lines[start + 1:]:
m = re.match(r'^ ([A-Za-z0-9_-]+):\s*$', l)
if m:
flush(); job, buf = m.group(1), []
else:
buf.append(l)
flush()
print(' '.join(f'{w}:{s}' for w, s in sorted(need) if s not in para or w not in para))
PY
)"
if [ -z "$SECTION" ]; then
broken V04 "every release-environment secret is named in README's Using a workflow section" "section not found"
elif [ -z "$MISSING" ]; then
pass V04 "every secret read in an 'environment: release' job is named, with its workflow, in README's Using a workflow section"
else
fail V04 "every secret read in an 'environment: release' job is named, with its workflow, in README's Using a workflow section" "missing: $MISSING"
fi
# V05: in docker.yml the Docker secrets are read only by the release-environment job.
if python3 -I - "$DW" <<'PY'
import re, sys
lines = open(sys.argv[1]).read().splitlines()
start = lines.index('jobs:')
jobs, job = {}, None
for l in lines[start + 1:]:
m = re.match(r'^ ([A-Za-z0-9_-]+):\s*$', l)
if m:
job = m.group(1); jobs[job] = []
elif job:
jobs[job].append(l)
readers = [j for j, b in jobs.items() if any('secrets.DOCKER_' in x for x in b)]
ok = readers and all(any(re.match(r'^\s+environment:\s*release\s*$', x) for x in jobs[j]) for j in readers)
sys.exit(0 if ok else 1)
PY
then pass V05 "docker.yml reads DOCKER_* only in jobs with 'environment: release'"
else fail V05 "docker.yml reads DOCKER_* only in jobs with 'environment: release'" "a reader is outside the release environment"; fi
# V06: the mapped-by-name path delivers the environment secret: release-workflows' own release.yml
# (which maps BLS_RELEASE_PRIVATE_KEY by name) ran 'check / credentials', an environment: release
# job that mints an App token from that key, successfully on stable (run 43443400874, v1.3.0).
J="$(gh api --method GET repos/blacklanternsecurity/release-workflows/actions/runs/43443400874/jobs \
--jq '.jobs[] | select(.name=="check / credentials") | .conclusion' 2>/dev/null)"
if [ -z "$J" ]; then skip V06 "mapped release-environment secret reaches the environment job (run 43443400874)" "jobs read failed"
elif [ "$J" = "success" ]; then pass V06 "mapped release-environment secret reaches the environment job (run 43443400874 'check / credentials' success)"
else fail V06 "mapped release-environment secret reaches the environment job (run 43443400874)" "conclusion=$J"; fi
# V10 (challenge C5, assumption A1): the unmapped case. At 5c7ee0e (the first release) release.yml
# mapped no secrets into release-check.yml, whose 'credentials' job runs in 'environment: release'.
# Desired property for A1: that job failed, at the App-token step (the key was empty).
RID="$(gh api --method GET 'repos/blacklanternsecurity/release-workflows/actions/workflows/release.yml/runs?head_sha=5c7ee0e77864ede3bfe468b721c126cfe6bd6408' --jq '.workflow_runs[0].id' 2>/dev/null)"
JC=""; JS=""
if [ -n "$RID" ] && [ "$RID" != "null" ]; then
JC="$(gh api --method GET "repos/blacklanternsecurity/release-workflows/actions/runs/$RID/attempts/1/jobs" --jq '.jobs[] | select(.name=="check / credentials") | .conclusion' 2>/dev/null)"
JS="$(gh api --method GET "repos/blacklanternsecurity/release-workflows/actions/runs/$RID/attempts/1/jobs" --jq '.jobs[] | select(.name=="check / credentials") | [.steps[] | select(.conclusion=="failure") | .name] | join(",")' 2>/dev/null)"
fi
if [ -z "$JC" ]; then skip V10 "unmapped release-environment secret is empty in the environment job (first release, 5c7ee0e)" "run or job read failed"
elif [ "$JC" = "failure" ]; then pass V10 "unmapped release-environment secret is empty in the environment job (run $RID 'check / credentials' failed at step: $JS)"
else fail V10 "unmapped release-environment secret is empty in the environment job (first release, 5c7ee0e)" "run $RID conclusion=$JC"; fi
# V11 (challenge C2, context for F1's basis): the commit that fixed the App key's description
# (2ecc0d1) also fixed docker.yml's copy. FAIL means a copied description was missed, which is how
# this PR's bug arose.
if git cat-file -e 2ecc0d1^{commit} 2>/dev/null && ! git show 2ecc0d1:.github/workflows/docker.yml | grep -qi 'never passed by a caller'; then
pass V11 "2ecc0d1 updated every copy of the 'Never passed by a caller' description"
elif git cat-file -e 2ecc0d1^{commit} 2>/dev/null; then
fail V11 "2ecc0d1 updated every copy of the 'Never passed by a caller' description" "docker.yml at 2ecc0d1 still says it"
else skip V11 "2ecc0d1 updated every copy of the description" "commit not in clone"; fi
# V12 (challenge C9): a repository test keeps the README's release-environment secrets and the
# copied descriptions in step with the workflows.
# A sync test reads the workflows, the README and the secrets together in one file.
SYNC=""; for t in tests/*.py; do [ -f "$t" ] || continue
grep -q '\.github/workflows' "$t" && grep -q 'README' "$t" && grep -qE 'secrets|environment' "$t" && SYNC="$SYNC $t"; done
if [ -n "$SYNC" ]; then
pass V12 "tests/ has a check tying release-environment secrets to README and the descriptions"
else
fail V12 "tests/ has a check tying release-environment secrets to README and the descriptions" "no file in tests/ reads .github/workflows, README and secrets together"
fi
summary
Docs only.
docker.ymldescribedDOCKER_USERNAMEandDOCKER_TOKENas "Never passed by a caller". That's wrong: likeBLS_RELEASE_PRIVATE_KEY, they're read in arelease-environment job, which receives an environment secret only when the caller maps it by name. Following the old description, asndb-server's and bbot'spublish.ymlleft the mapping out, and their first release would have stopped at the Docker Hub login with empty credentials (found in review on asndb-server#101 and bbot#3582; both callers are fixed).docker.yml: both secrets now carry the same description asBLS_RELEASE_PRIVATE_KEYinrelease-check.ymlandpublish.yml.docker.yml's two secrets alongside the App key.No behaviour change. actionlint 1.7.12 with shellcheck 0.11.0, and zizmor 1.30.1 online, report nothing.